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Q It is an efficient and effective strategy to utilize the nuclear norm approximation 
fT} ' to learn low-rank matrices, which arise frequently in machine learning and com- 

\ puter vision. So the exploration of nuclear norm minimization problems is gaining 

much attention recently. In this paper we shall prove that the following Low-Rank 
Representation (LRR) J2] Q~) problem: 

min||Z||„, s.t., X = AZ, 

. Z 
O . has a unique and closed-form solution, where X and A are given matrices. The 

proof is based on proving a lemma that allows us to get closed-form solutions to a 

\ category of nuclear norm minimization problems. 

> 
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In real applications, our observations are often noisy, or even grossly corrupted, and some observa- 
tions may be missing. This fact naturally leads to the problem of recovering a low-rank matrix X° 
from a corrupted observation matrix X = X° + E° (each column of X is an observation vector), 
with E° being the unknown noise. Due to the low-rank property of X°, it is straightforward to 
consider the following regularized rank minimization problem: 



min rank(D) +AII-BIL, s.t. X = D + E. 
d,e 1 



where A > is a parameter and ||-||^ is some kind of regularization strategy, such as the ^i-norm 
adopted by [3] HI, for characterizing the noise E°. As a common practice in rank minimization 
problems, one could replace the rank function with the nuclear norm, resulting in the following 
convex optimization problem: 

min IIDII +AIIJ5IL, s.t. X = D + E. (1) 

D,E 

The minimizer D* (with respect to the variable D) gives a low-rank recovery to the original data 
X°. The above formulation is adopted by the recently established Robust PCA (RPCA) method 
IfJJIU, which uses the ^i-norm to characterize the noise. However, such a formulation implicitly 
assumes that the underlying data structure is a single low-rank subspace. When the data is drawn 
from a union of multiple subspaces, denoted as Si, S2, • • • ,Sk, the PRCA method actually treats 
the data as being sampled from a single subspace defined by S = JD i=1 The specifics of the 
individual subspaces are not well considered, so the recovery may be inaccurate. 

To better handle the mixed data, in |f2j Q] we suggest a more general rank minimization problem 
defined as follows: 

min rank(Z) + A \\E\\ £ , s.t. X = AZ + E, 



'The content of this paper is a part of 1 1 1 
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where A is a "dictionary" that linearly spans the data space. By replacing the rank function with the 
nuclear norm, we have the following convex optimization problem: 

min \\Z\\ m + X\\E\\ t , s.t. X = AZ + E. (2) 

After obtaining an optimal solution (Z*,E*), we could recover the original data by using AZ* 
(or X - E*). Since rankLAZ*) < rank(Z*), AZ* is also a low-rank recovery to the original 
data X°. By setting A = I, the formulation (0 falls backs to (0. So our LRR method could be 
regarded as a generalization of RPCA that essentially uses the standard basis as the dictionary. By 
choosing an appropriate dictionary A, as shown in J2][T1, the lowest-rank representation also reveals 
the segmentation of data such that LRR could handle well the data drawn from a mixture of multiple 
subspaces. 

For ease of understanding the LRR method, in this work we consider the "ideal" case that the data 
is noiseless. That is, we consider the following optimization problem: 

min ||Z||„ , s.t. X = AZ. (3) 

We will show that this optimization problem always has a unique and closed-form minimizer, pro- 
vided that X = AZ has feasible solutions. 

2 A Closed-Form Solution to Problem © 

The nuclear norm is convex, but not strongly convex. So it is possible that problem (0 has multiple 
optimal solutions. Fortunately, it can be proven that the minimizer to problem (0 is always uniquely 
defined by a closed form. This is summarized in the following theorem. 

Theorem 2.1 (Uniqueness) Assume A ^ and X = AZ have feasible solutions, i.e., X G 
span (A). Then 

Z* = VAiVjVA)-^, (4) 

is the unique minimizer to problem (0, where Vx and Va are calculated as follows: Compute the 
skinny Singular Value Decomposition (SVD) of '[X , A], denoted as [X,A] = UT,V T , and partition 
V as V = [V X ] V A ] such that X = ITEVg and A = UY>V A T ■ 

From the above theorem we have the following two corollaries. First, when the matrix A is of full 
row rank, the closed-form solution defined by (0 can be represented in a simpler form. 

Corollary 2.1 Suppose the matrix A has full row rank, then 

Z* = A T (AA T y 1 X, 

is the unique minimizer to problem (0, where A T (AA T )~ 1 is the generalized inverse of A. 

Second, when the data matrix itself is used as the dictionary, i.e., A — X, the solution to problem 
(0 falls back to the outputs of a factorization based method. 

Corollary 2.2 Assume X ^ 0. Then the following optimization problem 

mm \\Z\\„, s.t. X = XZ, 

has a unique minimizer 

Z* = SIM(X), (5) 

where SIM(X) = VxV^ is called the Shape Interaction Matrix (SIM) ^5]/ in computer vision and 
X = IJx^xVx is the skinn y SVD °f x - 

The proof of Theorem |2.1| is based on the following three lemmas. 
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Lemma 2.1 Let U, V and M be matrices of compatible dimensions. Suppose both U and V have 
orthogonal columns, i.e., U T U = I and V T V = l3 then we have 

\\M\l = \\UMV T \U. 

Proof Let the full SVD of M be M = U m ^mV^, then UMV T = (UU m )^m{VV m ) t ■ As 
(UU M ) T {UU M ) = I and (VV M ) T (VV M ) = I, (UU m )^m(V m V) t is actually the SVD of 
UMV T . By the definition of the nuclear norm, we have ||Af|| t = tr(Sjw) = ||?7M1/ T || . 



Lemma 2.2 For any four matrices B, C, D and F of compatible dimensions, we have 

B C 
D F 



> \\B\ 



where the equality holds if and only if C = 0, D = and F = 0. 
Proof Lemma 10 of [6] directly leads to the above conclusion. 

Lemma 2.3 Let U, V and M be given matrices of compatible dimensions. Suppose both U and V 
have orthogonal columns, i.e., U T U — I and V T V = I, then the following optimization problem 



mm 

z 



\Z\ 



s.t. U 1 ZV = M, 



(6) 



has a unique minimizer 



Z* = UMV 1 



Proof First, we prove that ||Af|| # is the minimum objective function value and Z* — UMV T is a 
minimizer. For any feasible solution Z, let Z = Uz^z^z ^ e i ts SVD. Let B — U T Uz and 
C = VzV. Then the constraint U T ZV = M is equal to 

BT, Z C = M. (7) 
Since BB T = I and C T C = I, we can find the orthogonal complements^ B± and C± such that 

B 



B i 



and [C, CJ 



are orthogonal matrices. According to the unitary invariance of the nuclear norm, Lemma UM and 
(0, we have 



\Z\\ 



IS; 



B 

B± 



Z Z [C,C A 



B^zC BT, Z C± 
B\Y*zC B±Y,zC± 



> \\BXzC\l = \\M\l , 
Hence, ||M|| t is the minimum objective function value of problem ©. At the same time, Lemma 
Oproves that \\Z*\\ # = ||[/My T ||^ = ||M||^. So Z* = UMV T is a minimizer to problem ©. 

Second, we prove that Z* — U MV T is the unique minimizer. Assume that Z\ = UMV T + H is 
another optimal solution. By U T Z\V — M, we have 

U T HV = 0. (8) 

Since U T U = I and V T V = I, similar to above, we can construct two orthogonal matrices: [U, U±] 
and [V, V±}. By the optimality of Z%, we have 



Ml 



\Zi\ 



UMV 1 



H 



WML. 



(UMV 1 +H)[V 1 V 1 _ 



M U t HVjl 
UlHV UlHV±_ 

According to Lemma lZ2l the above equality can hold if and only if 

U T HVx = UlHV = UlHV±_ = 0. 

Together with ((H), we conclude that H = 0. So the optimal solution is unique. 

2 Note here that U and V may not be orthogonal, namely, UU T 7^ I and VV T 7^ I. 

3 When B and/or C are already orthogonal matrices, i.e., B± — and/or C± = 0, our proof is still valid. 
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The above lemma allows us to get closed-form solutions to a class of nuclear norm minimization 
problems. This leads to a simple proof of Theorem l2.ll 

Proof (of Theorem \2.1\ Since X 6 span (A), we have rank ([X, A]) = rank (A). By the defini- 
tions of Vx and Va (see Theorem |2.1| ), it can be concluded that the matrix Vj has full row rank. 
That is, if the skinny SVD of Vj is Ui'SiV^, then U\ is an orthogonal matrix. Through some 
simple computations, we have 

VaIvJVa)- 1 = V^Uj. (9) 

Also, it can be calculated that the constraint X = AZ is equal to V£ = Vj Z, which is also equal 
to Y,^ 1 UiVx = Vi Z. So problem (f3]) is equal to the following optimization problem: 

min ||Z||„, s.t. V^Z = T^ 1 UjVx- 

By Lemma |2~3l and (O, problem (f3]l has a unique minimizer 

z* = Vx^ufvZ = v A (yjEv A )- 1 v$. 

3 Conclusion 

In this paper, we prove that problem (O has a unique and closed-form solution. The heart of the 
proof is Lemma 12.31 which actually allows us to get the closed-form solutions to a category of 
nuclear norm minimization problems. For example, by following the clues presented in this paper, 
it is simple for one to get the closed-form solution to the following optimization problem: 

mm \\Z\\„, s.t. X = AZB, 

where X, A and B are given matrices. Our theorems are useful for studying the LRR problems. For 
example, based on Lemma |2~3l we have devised an method to recovery the effects of the unobserved, 
hidden data in LRR Q. 
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